This invention relates generally to the encoding and decoding of multimedia data, and more particularly the invention relates to a decoder of audio and video data which has been encoded in accordance with the MPEG (Motion Picture Experts Group) standard for full-motion video.
The MPEG decoding algorithm specifies several buffers for proper decoding. The first type of buffers are coded bitstream buffers. If the decoder decodes video only, then one coded bitstream buffer is needed. If the decoder decodes the multiplexed system bitstreams, then the number of coded bitstream buffers needed is equal to the number of bitstreams synchronized by the decoder. The second type of buffers are decoded picture buffers used as reference data in the decoding process. Two picture buffers are needed for this purpose. When the coded pictures are progressive (as in the case in MPEG 1 and some subsets of MPEG 2) and the decoder has to support conversion of the decoded picture to interlaced display, at least a third picture buffer is needed.
Even for constrained MPEG I video bitstreams, the size of the needed coded video bitstream buffer (typically about 40 Kbytes) and SIF size picture buffers (typically about 125 Kbytes per picture) preclude a cost effective solution that supports the needed buffers inside the decoder. An external buffer completely controlled by the decoder is a better solution.
Of the common types of RAM devices (SRAM, VRAM and DRAM), the DRAM offers the most cost effective solution and indeed many of the decoders already implemented use external DRAM buffers. The requirements of the DRAM structure and mapping of the various buffers to the DRAM address space are described in copending application Ser. No. 08/245,465 filed May 18, 1994 for DYNAMIC RANDOM ACCESS MEMORY FOR MPEG DECODING.
The DRAM has many "customers" within the decoder device (i.e., writing of the data into the various buffers as this data becomes available and reading the data of the various buffers, sometimes for different purposes, as the data is needed). The present invention is directed to the requirements of the DRAM and the MPEG decoding algorithm as to when and for how long to allocate the DRAM for each of its "customers".
To make the explanation easier, the term "slot" will be used, where a slot is a time period during which the DRAM is either written to or read from a specific buffer for a specific purpose.
Consider now MPEG and DRAM requirements as illustrated in FIG. 1. Most DRAMs have a special "page mode" read or write where consecutive transfer of cells within the same row is much faster (about three times typically) than a transfer of a random single cell. There is some overhead involved so that the larger the "page" (number of cells of the same row transferred consecutively), the smaller is the average time per transferred cell. A slot can contain more than one page.
On the other hand, if data is not written "just in time" after it becomes available, or not read "just in time" before it is needed, it will have to be stored temporarily in buffers on board the decoder device. The larger the delay, the larger the needed buffer.
Video coded data becomes available as it enters the decoder. The data can enter the decoder at a constant bit rate or by demand. It is needed before the decoding of each header and each sample block of each of the components. The amount of data needed by each header or block is variable. See FIG. 1 for an overview block diagram.
Serial coded data (audio or private) is available as it enters the decoder. The data can enter the decoder at a constant bit rate or by demand. It is needed in a constant bit rate expected by its receiver (and specified (for audio) in the coded bitstream).
A decoded picture is composed of three rectangular components: one (the Y component) is l lines by p samples by 8 bits, and the other two (the U and V components) are 1/2 line by p/2 samples by 8 bits. The pictures are written in 8*8 sample blocks as they are decoded. The order of decoding is by macroblocks which contain four Y blocks followed by one U block and then one V block. For some macroblocks, decoding requires reference data from one reference picture. For some macroblocks, decoding requires reference data from two reference pictures. The data needed for the decoding of each block of those macroblocks is one 9*9 sample block with origin at any sample of the component, from either one or both of the reference pictures.
On the average, the amount of coded data per block is decreasing as the number of reference pictures used (0, 1 or 2) is increasing. For display, each of the three picture buffers (or only two, as the case may be), is read in raster scan order. The data of all three components is usually needed in parallel. The DRAM requires a periodic refresh operation of each of its rows. This refresh is automatically done with each transfer operation. Otherwise, it requires a special operation.